Gödel Agents in a Scalable Synchronous Agent Framework

A synchronous framework for the interaction of an agent and an environment based on Moore machines is introduced. Within this framework, the notion of a Gödel agent is defined relative to a family of agents and environments and a time horizon T . A Gödel agent is the most flexible, adapting and self-improving agent with regard to the given environment family. It scores well across many environments, and not only in a selected few. Ideas from infinite game theory and ruin theory are used to get well-defined limits for T → ∞ by introducing negative goals or repellors. This allows to score actions of the agent by how probable an action makes the survival of the agent till the end of time. Score functions of this type will be called “liveness” scores, and they provide a solution to the horizon problem from a foundational point of view. Additionally, by varying the agent and environment families, one gets a scalable and flexible testbed which could prove to be well-suited for analyzing phenomena of adaptation and self-improvement, both theoretically and empirically. 1 A Scalable Synchronous Agent Framework Theoretical investigations have to be conducted within a conceptual framework. The process of taking a notion of colloquial language and turn it into a formal, precisely defined one often is not a straight path from the colloquial notion to the formal one, but a long and intertwined development resulting in several precise, but different versions of the colloquial term. These differences often are very subtle, but can have profound implications for the results obtainable within the respective frameworks. This conceptual dynamics also holds for the notion of an agent, which plays a central role in computer science, but especially in artificial intelligence. The agent concept underlying much of the research in foundations of artificial intelligence is, for example, defined by M. Hutter in [5], p. 126. It consists of two interacting Turing machines, one representing the agent and one representing the environment. If the environment produces an output, it is written on the percept tape of the agent. Then the agent starts its computations, deliberating the new percept, and finally produces an action as output, which is written on the action tape of the environment. While one machine is computing its next output, the other one is effectively suspended.